Abstract


Bias can occur in any phase, component, or part of a research. A thorough understanding of biases and how it can affect the results is essential to produce high-quality research. This narrative review provides an overview of the topic of research bias with a focus on publication bias, its effects, and ways to avoid it.



1. Introduction


Expert Researchers very well know that biases can creep its way into any research study — it’s naïve to think that a research study is 100 percent fair and completely free from biases. Bias can occur in any phase of research — planning, recruiting, data collection, analysis, and publication. There are multiple sources of Biases. It can come from the researcher/experimenter, subjects/participants, raters, and even the questions/instrument used (Sarniak, 2015). No matter how real the scenario, task, data, software, or environment is, these methods are feigned, and subtle biases will always creep into the study.

Why is it important to understand research biases? From a reader’s point of view, understanding research bias allows them to critically and independently review the scientific literature. This, in turn, will help them analyze and decide if the literature, its results and inferences are relevant to their project (Pannucci, 2010).

From a researcher’s point of view, a thorough understanding of biases and how it can distort the results is essential to produce high-quality research. It is imperative that an experimental design process involves understanding the inherent biases and minimizing the effects.

There are many biases that can exists in a research study, for example, Hawthorne effect, task-selection bias, social desirability bias, sample bias, selection bias, primacy & recency effects, negativity bias, framing effect, anchoring bias, acquiescence bias, question-order bias, and many more. However, the focus of this paper is to understand publication bias, its effects, and ways to contain it.



2. What is Publication bias?


Most of the research that appears in the published literature is systematically unrepresentative of the population of completed studies (Rothstein et al., 2006). This is known as publication bias. In other words, not all studies or findings from the studies are published in a reviewed journal. The bias may be introduced intentionally or unintentionally, consciously or unconsciously, into the process of research dissemination

Easterbrook et al. (1991) found that studies with statistically significant results have a higher likelihood of being published than those with no difference between the study groups. Studies with significant results are also more likely to lead to a greater number of publications and presentations and to be published in journals with a high citation impact factor. Publication bias was also observed in studies with increasing sample size. Also, observational and laboratory-based experimental studies were more likely to be published that randomized clinical trials.

FPublication bias can be introduced by investigators/authors, journal editors, peer reviewers, and/or study sponsors. Song et al. (2010) surveyed 21 investigators/authors and found that the main reasons for non-publication of completed studies included lack of time or low priority (34.5%), unimportant results (19.6%), and journal rejection (10.2%). Firm editorial policies and lack of editorial independence can result in the introduction of publication bias from editor’s end (Song et al., 2013). Sponsors’ commercial interest too can lead to publication bias. They can suppress the publication of a study or influence the researchers to report selected or skewed results (Song et al., 2013).

As cited by Toews et al. (2017), the other sources of publication bias include:
1. Time-lag bias — Less noteworthy findings take more time to get published.
2. Language bias — Studies showing significant findings are more likely to be published in the English language in international journal
3. Grey literature bias — Studies with less noteworthy findings get published in outlets other than peer-reviewed journals
4. Truncation bias — Only selective findings are published in peer-reviewed journals due to the prescribed word limit.



3. Why it matters?


3.1. Publication bias affects research validity


The studies conducted by Toews et al. (2016) and Schmucker et al. (2014) concluded that the substantial proportion of non-dissemination may impact healthcare research, practice, and policy. Schmucker et al. (2014) mention that “...the validity of systematic reviews is threatened if journal publications represent a biased selection of all studies that have been conducted…”.

Eyding et al. (2010) performed a Systematic review and meta-analysis, including unpublished results, to access the benefits and harms of the drug Reboxetine, a commonly prescribed antidepressant; and to measure the impact of potential publication bias in trials of Reboxetine. Unfortunately, the results revealed that the beneficial effect of Reboxetine was based on partial data that comprised findings from only 26% of the sample. Data for the remaining 74% of the patients were not published in the primary studies. The review authors replicated the analysis with published and unpublished data and concluded that Reboxetine is, overall, an ineffective and potentially harmful antidepressant; and that published evidence was affected by publication bias.


3.2. Non-dissemination isolates knowledge


A study conducted by Toews et al. (2016) with 859 researchers revealed that 68.1% of researchers had conducted at least one qualitative study that they had not published in a peer-reviewed journal. According to the result, top reasons for this was publication being intended (35.7%), resource constraints (35.4%), and that the authors withdrew after the paper was rejected by one or more journals (32.5%). Similar results were observed by Schmucker et al. (2014). After evaluating 23 publications, the study found that only 46.2% of the studies approved by research ethics committees were published.

Because of non-dissemination, many statically insignificant, but critical and relevant findings remain inaccessible and isolated.


3.3. Distortion of results in meta-analysis


While conducting systematic reviews of quantitative studies with meta-analyses, non-dissemination can lead to inadequacy of data. This deficit can, in turn, lead to imprecision of pooled estimates i.e. Overall variance, standard deviation, mean (Toews et al., 2017). In other words, publication bias distorts the results of meta-analyses and systematic reviews as the published studies are no longer a representative sample of the available evidence.


3.4. HARKing


Many researchers have argued that competition in science contributes to misinterpretation and distortion of results. The desire to find a positive outcome can lure authors to tweak the hypothesis in order to better suit their data. This is known as HARKing — hypothesizing after the results are known. HARKing involves careful examination of the data that does not fit into the tested hypothesis. The authors may then choose to partially or completely ignore this data. There are numerous reports of scientific misconduct where researchers have completely forged the published data (Mlinarić, 2017).


3.5. Publication bias is on the rise


Publication bias has been under scrutiny in the literature for decades now. However, there is evidence that suggests that this bias has been increasing in recent years. Fenelli (2011) investigated over 4600 publications from different countries and fields to uncover strong evidence for a steady and significant increase in publication bias over the years. According to this study, the overall frequency of positive supports has grown by over 22% between 1990 and 2007 (n = 4656, p < 0.001). The percentage varies significantly for various disciplines and countries.



4. How to contain publication bias?


4.1. Pre-registration of research protocols


One way to contain publication bias is by adhering to enhanced research standards and by reducing prejudices (Ioannidis, 2005). Enhanced research standards include techniques like pre-registration of research protocols/trials and registration of data collection. The responsibility of containing publication bias also falls on the shoulders of the publication houses. Publication houses can help reduce publication bias by making pre-registration mandatory and by focusing on the development of comprehensive, computerized databases to promote transparency (Abaid et al, 2007).


4.2. Identifying and including unpublished studies


Publication bias in systematic reviews and meta-analysis can be reduced by including unpublished studies and unpublished outcomes of published studies. This will help provide a better estimate of effectiveness or association (Song, 2013). However, locating unpublished studies or unpublished outcomes of published studies can be a challenge. According to Young and Hopewell (2009), the best way to find unpublished outcomes of published studies is to contact the authors of the missing data. Grey literature can be found in databases dedicated to grey literature, theses, and dissertations (Cornell University Library).



5. Conclusion


The responsibility of containing publication bias rests on the shoulders of every actor associated with the research study. It is very imperative that best practices and high standards are followed by everyone involved in the process of initiating, conducting and disseminating a study.



References


A Guide to Conducting Systematic Reviews: Searching the Gray Literature. (n.d.). Retrieved March 21, 2018, from http://guides.library.cornell.edu/c.php?g=459012&p=3142103

Abaid, L. N., Grimes, D. A., & Schulz, K. F. (2007). Reducing publication bias through trial registration. Obstetrics & Gynecology, 109(6), 1434-1437.

Easterbrook, P. J., Gopalan, R., Berlin, J. A., & Matthews, D. R. (1991). Publication bias in clinical research. The Lancet, 337(8746), 867-872.

Eyding, D., Lelgemann, M., Grouven, U., Härter, M., Kromp, M., Kaiser, T., ... & Wieseler, B. (2010). Reboxetine for acute treatment of major depression: systematic review and meta-analysis of published and unpublished placebo and selective serotonin reuptake inhibitor controlled trials. Bmj, 341, c4737.

Fanelli, D. (2011). Negative results are disappearing from most disciplines and countries. Scientometrics, 90(3), 891-904.

Ioannidis, J. P. (2005). Why most published research findings are false. PLoS medicine, 2(8), e124.

Mlinarić, A., Horvat, M., & Šupak Smolčić, V. (2017). Dealing with the positive publication bias: Why you should really publish your negative results. Biochemia Medica, 27(3), 030201. http://doi.org/10.11613/BM.2017.030201

Pannucci, C. J., & Wilkins, E. G. (2010). Identifying and Avoiding Bias in Research. Plastic and Reconstructive Surgery, 126(2), 619–625. http://doi.org/10.1097/PRS.0b013e3181de24bc

Rothstein, H. R., Sutton, A. J., & Borenstein, M. (Eds.). (2006). Publication bias in meta-analysis: Prevention, assessment and adjustments. John Wiley & Sons.

Sarniak, R. (2015, August). 9 types of research bias and how to avoid them. Retrieved March 19, 2018, from https://www.quirks.com/articles/9-types-of-research-bias-and-how-to-avoid-them

Schmucker, C., Schell, L. K., Portalupi, S., Oeller, P., Cabrera, L., Bassler, D., ... & Meerpohl, J. J. (2014). Extent of non-publication in cohorts of studies approved by research ethics committees or included in trial registries. PloS one, 9(12), e114023.

Song, Fujian, et al. "Dissemination and publication of research findings: an updated review of related biases." Health Technol Assess 14.8 (2010): 1-193.

Song, F., Hooper, L., & Loke, Y. (2013). Publication bias: what is it? How do we measure it? How do we avoid it?. Open Access Journal of Clinical Trials, 2013(5), 71-81.

Toews, I., Glenton, C., Lewin, S., Berg, R. C., Noyes, J., Booth, A., … Meerpohl, J. J. (2016). Extent, Awareness and Perception of Dissemination Bias in Qualitative Research: An Explorative Survey. PLoS ONE, 11(8), e0159290. http://doi.org/10.1371/journal.pone.0159290

Toews, I., Booth, A., Berg, R. C., Lewin, S., Glenton, C., Munthe-Kaas, H. M., ... & Meerpohl, J. J. (2017). Further exploration of dissemination bias in qualitative research required to facilitate assessment within qualitative evidence syntheses. Journal of clinical epidemiology, 88, 133-139.

Young, T., & Hopewell, S. (2009). Methods for obtaining unpublished data. Cochrane Database of.